To meet the demand for wireless data traffic having increased since deployment of 4th generation (4G) communication systems, efforts have been made to develop an improved 5th generation (5G) or pre-5G communication system. Therefore, the 5G or pre-5G communication system is also called a ‘Beyond 4G Network’ or a ‘Post Long Term Evolution (LTE) System’.
The 5G communication system is considered to be implemented in higher frequency (mmWave) bands, e.g., 60 GHz bands, so as to accomplish higher data rates. To decrease propagation loss of the radio waves and increase the transmission distance, the beamforming, massive multiple-input multiple-output (MIMO), Full Dimensional MIMO (FD-MIMO), array antenna, an analog beam forming, large scale antenna techniques are discussed in 5G communication systems.
In addition, in 5G communication systems, development for system network improvement is under way based on advanced small cells, cloud Radio Access Networks (RANs), ultra-dense networks, device-to-device (D2D) communication, wireless backhaul, moving network, cooperative communication, Coordinated Multi-Points (CoMP), reception-end interference cancellation and the like.
In the 5G system, Hybrid frequency shift keying (FSK) and quadrature amplitude modulation (FQAM) and sliding window superposition coding (SWSC) as an advanced coding modulation (ACM), and filter bank multi carrier (FBMC), non-orthogonal multiple access (NOMA), and sparse code multiple access (SCMA) as an advanced access technology have been developed.
Communication systems have assumed an interference signal as a Gaussian distribution to operate (adaptive modulation and encoding operation, soft-decision decoding metric generation, etc.) the system with low complexity. Owing to this, the communication systems have mainly used a Quadrature Amplitude Modulation (QAM)—series modulation scheme in order to make a characteristic of the interference signal maximally close to a Gaussian model. Also, the communication systems used a scheme of achieving target error performance by repeatedly transmitting QAM symbols to terminals that are not able to satisfy the target error performance even if applying a minimum channel code rate and a minimum modulation order.
However, in a recent wireless communication network, it was proved that a case where a statistical distribution of an additive noise follows a Gaussian distribution is the worst case in view of a channel capacity. Accordingly, it is obvious that, if a statistical distribution of interference signals having a characteristic of the additive noise follows a non-Gaussian distribution, it may get a higher network throughput than a conventional system.
A modulation scheme proposed for this reason is Frequency and Quadrature—Amplitude Modulation (FQAM). The FQAM is a hybrid modulation scheme in which QAM and Frequency-Shift Keying (FSK) are combined and only some of many subcarriers configuring a symbol are activated and therefore, a statistical distribution of an interference signal has a non-Gaussian characteristic.
This has similarities with a conventional FSK modulation scheme, but the FQAM transmits a QAM symbol to an activated subcarrier, thereby being capable of greatly improving a spectral efficiency than the FSK scheme.
If the FQAM is applied to users of a cell boundary where an interference signal is very strong, a non-Gaussian interference channel may be formed. Also, the FQAM repeatedly transmits the QAM symbol, thereby being capable of very greatly improving a network throughput compared to a system that forms a Gaussian interference channel. To apply the modulation scheme such as the FQAM and achieve performance improvement, it is essential to apply a non-binary encoding/decoding technology. However, the non-binary encoding/decoding technology has a problem in which complexity is very large.